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November 10, 2015 
Dear Reader, 


In the following package of material, you will find documentation outlining our planning 
efforts for the HUIT Cloud & DevOps program. The material is intended to provide a 
high-level, strategic outline of our approaches for the following five areas: 


* Overview: The Plan for the Plan 
* Current-State Services 

¢ Future-State Services 

* Service Transition 

* Cloud Migration 


In total, these documents explain the “case for the cloud” for HUIT, and review the service, 
technology, process, and people changes that are necessary for successful implementation. 
The next step associated with the effort is to develop a coordinated implementation, 
staffing, and change management plan that maps process, dependencies, and delivery 
milestones for impacted organizations. 


Thank for you taking the time to review our strategy. 


Kind Regards, 
Cloud & DevOps Working Group and Program Team 


CLOUD & DEVOPS 50 Church Street, Third Floor, Cambridge, MA 02138 
EMAIL cloud-help@ harvard.edu WEB cloud.huit.harvard.edu 
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Cloud & DevOps ‘Plan for the Plan’ 
The Case for Cloud & DevOps 


Agenda 


e Purpose and Intended Outcomes 

e About Cloud & DevOps 

e What is the Cloud? 

e Value of the Cloud 

e What is DevOps? 

e Value of DevOps 

e Getting There from Here 

e Service Transition: Infrastructure and Application Services 
e Migration Approach 

e Managing Change 


Purpose and Intended Outcomes 


Purpose 


To summarize cloud computing and DevOps methodologies, explain their 
value, and examine how cloud and DevOps can provide value and 
efficiencies within HUIT. 


Intended Outcomes 
e Knowledge of cloud and DevOps key concepts 
e Awareness of the potential for value offered by cloud and DevOps 
e Understanding of a migration approach and service transition plan within 
HUIT, as well as critical aspects of change management 


About Cloud & DevOps 


Cloud computing and DevOps methodologies are two major IT industry- 
wide transformations that have been underway for many years. 


These two trends have an impact on the way we build and/or operate 
applications: 


e This affects the work we do in application groups, such as ATS 
e This impacts large portions of our infrastructure services 


We can either ... 
e Embrace these trends and capitalize on the benefits that they offer 
e Hang on to our current practices for as long as we can 


About Cloud & DevOps 


Both cloud and DevOps each have value on their own, but the value of the 
two together is greater than the sum of the individual parts. 


Ultimately, the value of cloud and DevOps working together lies in: 
e A reduction in operating costs 


e Increased speed, flexibility, and reliability 


What is the Cloud? 


“The Cloud” incorporates two 
fundamental ideas: 


“Off-Premise” 
e Using others’ data centers, servers, etc. 


Advanced Capabilities 
e Access to new technical capabilities not 
available in traditional data centers that 
deliver higher value to Harvard 


There are two key cloud models to highlight: 


Infrastructure as a Service (laaS) 
e Servers in the cloud (e.g. AWS) 


Software as a Service (SaaS) 


e Applications in the cloud (e.g. Facebook, 
Office 365, Gmail) 


(J Office 365 
Gmail 


An Analogy: Digital Photography 


Cloud computing is to data-center computing as film photography is to 
digital photography ... 


Film Camera Digital Camera 
The good: Camera lens, camera The good: Camera lens, camera 
functionality functionality 
The bad: Can't edit, each print costs The better: On-the-fly editing, delete 
money, each print takes time bad prints, immediate viewing 


... and similarly, the cloud introduces on-demand self-service, rapid 
elasticity, and improved integration. 


Value of the Cloud 


Cost 
e Some reduction in operating cost as part of the transition 
e Cloud costs are directly tied to usage — today, we often pay for peak 
usage rather than actual usage 
Speed 
e Ability to meet new or changing needs in minutes or hours, 
not days or weeks 
Flexibility 
e Ability to adjust to changing conditions such as seasonal spikes, 
or extreme conditions and other disasters 


Reliability 


e Ability to create systems that are more resilient, secure, and standardized 


What is DevOps? 


“DevOps” means applying three important concepts that change the way 
applications are developed and operated. 


Agile 
e Deliver value fast and frequently 
e Correct the course if necessary 
e Understand risk and challenges more quickly 
e Reduce wasted efforts 


Automation 
e Use tools in order to remove as much human-factor risk as possible 


Integrated, multidisciplinary teams 


e Teams structured to include all skills necessary to develop and operate 
a system 


What is DevOps? 


The current model: 


Org 1 


Bus. Analyst Developer 


The DevOps model: 


Integrated Service Team C ) 


Bus. Developers DevOps 
Analysts Engineers C Agile 


Value of DevOps 


Cost 
e Team integration means less redundant management 
e Labor cost goes down as a result of automation 
Speed 


e Agile methodologies and automation dramatically increase the speed of 
delivery 


e Integrated teams mean faster and more efficient collaboration 
Flexibility 

e Agile methodologies and automation allow for quicker adjustments 
Reliability 


e Automation removes a category of human errors while promoting or even 
forcing standardization 


e Integrated teams are more self-sufficient and accountable 


Getting There from Here 


Current State Future State 


New 


Application Application 
Services Services 
Infrastructure New 


Services Operations 


Service Transition: Infrastructure 
Infrastructure services will evolve, remain the same, or be deprecated. 


Current Services Future Services 


Crimson Cloud 
Storage as a Service 


Data Protection 
Snapshots, Self-Healing 


Networking 
+ Increased Extranet Connectivity 


Crimson Cloud 
laaS + PaaS 


Cloud Center of Excellence 


Migration Approach Summary 


There are two methods to migrate an application to the cloud as laaS: 
replatforming, or “lift and shift.” HUIT application teams can migrate via 
one of these options either through the Cloud & DevOps program, or on 
their own. 


Replatform Lift and Shift 


e Replaces application functionality e Copies the application “as is” 
with cloud services to the cloud 

e Requires developer commitment and e Requires research on existing 
cloud service expertise to implement connections through discovery 
application changes process 

e Operational support model shifts to e Enables fast migration timelines 
cloud program team and embedded using a migration toolset 
DevOps resource when staffed e Known operational support model 


leveraging the infrastructure team 


Managing Change 


Implementing organizational change of this magnitude without impacting 
service delivery requires a coordinated approach: 


Define Needs Identify Decide 
Options . 
* Problem * SLT Admin 
statement/ * Possibilities Subgroup 
opportunity * Dependencies *CIO 
0 Scope e Impact 
¢ Urgency 


Assess cultural considerations at each step 
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Cloud & DevOps ‘Plan for the Plan’ 
SOC, NOC, and ATS Current State 


Agenda 


e Purpose and Intended Outcomes 
e ATS Services: Current State 
o FAS and College Systems 
o Central Administration Systems 
o Office of President & Provost Systems 
o ATS Practices 
e Infrastructure Services: Current State 
o SOC Services 
o NOC Services 


Purpose and Intended Outcome 


Purpose 


To provide a financial and staffing overview of the current state of the SOC, 
NOC, and ATS. 


Intended Outcome 


e Understanding of current-state financial and staffing climate for the 
NOC, SOC, and ATS 


ATS Services: Current State 
Current State 


FAS and College 
Systems 


Application 
Services 


Central Administration 
Systems 


Inf t t Office of President & Provost 
ntrastructure Syn 


Services 


ATS 
Practices 


ATS Services: Current State 


Current State 


FAS and College Systems ATS Practices 
Aurora Athletics ServiceNow 
A li ti College FAS Admin Atlassian 
Pp hich lon GSAS App Arch Salesforce 
Services 
Central Administration Systems 
Release Mgt 
AA&D HR Finance Data & Arch 
Museum Student Financial 


Sponsored Research Data Mgt/Bl 


Office of President & Provost 


Infrastructure Office of the President & Provost 
Services Research Office 


NOC/SOC Services: Current State 


Current State 


Application 
Services 


NOC/SOC Services: Current State 


Current State 
= 


Systems Mgmt (Wintel/Linux) Operations and Production 
Storage and Archive Database and Application Management 


Data Center Networking Data Center Firewall 
Data Center Load Balancing Extranet Connectivity 
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Cloud & DevOps ‘Plan for the Plan’ 
Future-State Service Strategy 


Agenda 


e Purpose and Intended Outcomes 
e Drivers for Cloud Services 
e Cloud Services Future State 
e User Segments and Requirements 
e Crimson Cloud Service: laaS, PaaS, and StaaS 
e Cloud Center of Excellence 
e IT Provider Services 
o Network and Security 
o Billing and Account Management 
e Cloud Operations Services 
o Operations 
o Disaster Recovery 
e Application Team Impact 
e ITSM Impact 
e End-State Tools 


Purpose and Intended Outcomes 


Purpose 


Describe the future state for cloud services within HUIT, including proposed 
cloud services, the organization tasked with governing these services, and the 
extension of operations to support cloud services. 


Intended Outcomes 
e Understanding of the future state for HUIT cloud services 
e Awareness of the oversight organization and model for cloud services 
e Knowledge of the infrastructure and operations organization services 


Drivers for Cloud Services 


Illustrate an understanding of diverse user needs for cloud solutions 


e Create a suite of service offerings that are deliberately crafted based on 
known user needs and behaviors 


Provide value differentiation over “direct-to-vendor” offerings 
e Incorporate organizational best practices for security and networking into 
the delivery process for cloud resources, enabling teams who deploy to the 
cloud to navigate complex policies 
e Introduce automation to empower application teams to deliver functionality 
without needing in-depth understanding of a specific vendor 


Provide a means to estimate, trace, and recover costs in a meaningful way 


e Bundling and mapping costs for specific resources, tools, and technologies 
to services helps facilitate long-term financial planning and enables the 
organization understand TCO 


Define the transition from current-state services to future-state hosting 


e Clearly define and articulate how infrastructure hosting services will evolve 
with the introduction of the new cloud hosting delivery model 


Drivers for Cloud Services ... aS a Service 


All compute resources all of the way up Comput fe 
to the application level are managed by a in S managed remotely by a vendor 
you. a ered to on- automated worktlows. 
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Future-State Services 


FAS and College Systems ATS Practices 
Aurora Athletics Developer and 
College FAS Admin Sees 
Management 
GSAS Toolset 


Central Administration Systems 


AA&D HR Finance 
Museum Student Financial 


Sponsored Research Data Mgt/BI 


Office of President & Provost 


Office of the President & Provost 


Research Office 


All systems services remain, but are structured as 
integrated teams with embedded DevOps. 


Future State 


NEW 
Application 
Services 


NEW 
Infrastructure 
Services 


Future-State Services 


Crimson Cloud 


Infrastructure Platform Storage 
as a Service as a Service as a Service 


Cloud IT Provider Services 


Billing & Account 


Network & Security Management 


Cloud Operations Disaster Recovery 


Knowledge Sharing 


Standards & Best 
Practices 


Architecture & Design 


Future State 


NEW 
Application 
Services 


User Segments and Requirements 


Self- 
Sufficient 


Agility- 
Seeking 


Storage- 
Seeking 


Reliability- 
Seeking 


Little to no need for centralized service 
offerings. Has “full stack” dev team and/or 
skills to independently support the entire 
application development lifecycle. Team is 
familiar with or has potential to implement and 
support cloud offerings without assistance. 


Motivated by efficiency and quick delivery of 
resources. Seeks speed above all else and 
has a low tolerance for complex or 
cumbersome processes. 


Need to store large amounts of information and 
share it with others. Wants tiered storage 
capacity that is flexible and on demand to meet 
their ever-growing data needs. 


Prioritizes application availability. Demands 
stability and resiliency in the form of uptime, 
low RTO, and low RPOs. Examples include 
enterprise and mission-critical applications. 


Autonomy in creation, maintenance 
and support of apps and resources. 
Prefers access to advisory services 
and/or information to get reusable 
artifacts and design validation. 


Favors automation for provisioning and 
maintaining cloud resources. 


Seeks “ease of use” and self-service. 


Favors bundled options that are 
workflow-enabled and require minimal 
customization or ongoing support. 


Seeks file share options for sharing 
files across departments or groups. 


Needs flexible, cost-effective storage. 


Seek automated recovery options for 
compute and data. 

Require disaster recovery to ensure 
business functions available in a crisis. 


Crimson 
Cloud 
laaS 


Crimson 
Cloud 
PaaS 


Crimson 
Cloud 
Storage 


SaaS 


Cloud Services by User Segment 


Self-Sufficient _ Agility-Seeking Storage-Seeking _ 


¥v ¥ ¥ 


Crimson Cloud 


Infrastructure as a Service Platform as a Service Storage as a Service 


Managed Cloud Network & Security: Cloud Connectivity, VPC Management, Firewall-Controlled Access [__] Design Patterns 


[__] Cloud SMEs 


Billing & Account Management Service: Consolidated Billing, Account Management, IAM 


[rt DevOps Training 
& Support 


dec 
eu) 


Vendor 
Management 


Cloud Operations: [_]Monitoring [Auditing [_]Logging — [_] Scheduling 


; 
| Disaster Recovery Service: [| Cloud-to-Cloud [| On-Prem to Cloud [ ] Cloud Backup 


-[_] Knowledge Share 


Crimson Cloud: laaS 


Self-service capabilities enable users to rapidly provision 
resources; automation and standardized templates 
improve service quality and reduce time to delivery. 


Features 
e Fast, automatic resource generation 
e Tested templates/scripts from a central knowledge store 
e Full self-service access to provision/remove resources 


Target Audiences 

e Application development teams 

e Students and faculty 

Use Cases 
Faculty member hosting statistical probability app 
Student needing compute resources for class assignment 
Partner org hosting production app with periodic spikes 
App team needing dynamic storage to develop business- 
critical application 


Manage 


You 


r 


Crimson Cloud 


Applications 
Data 
Middleware 


O/S 


Servers 
he. £62) 


Storage 
ha $3) 


Network & Security 
(le VPC, DC, FW) 
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Crimson Cloud: PaaS 


Users can develop, deploy, and manage end-to-end cloud 
resources and supporting toolsets with no worry about 
underlying infrastructure. 


Features 
e Self-service access to a fully integrated cloud platform 
e SLA provided by direct from selected PaaS vendor 
e Centrally negotiated contract lowers costs, standardizes 
terms/conditions 
Target Audiences 
e Partner org with small, development-focused team 
e Faculty member hosting information for a class 
Use Cases 
e Partner org with app with constantly-changing content 
e Student building a website for a class 
e Partner org that can hire developers, but has difficulty 
finding qualified DevOps engineers 
e Deadline-driven team needing to reduce deployment time 


Crimson Cloud 


i Applications 

6 

SS Data 
Middleware 


ols 


Servers 
(ie EC2) 


Storage 
(be 53) 


Network & Security 
(ie. VPC, DC, FW) 
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Crimson Cloud: STaaS 


From application storage to file shares to archive cold storage, 
Crimson Cloud’s storage service will provide cost-effective, scalable, on- 
demand storage capabilities. 


Features 
e Multi-tier storage (EBS, S3, Glacier, etc.) 
e File sharing (NFS/CIFS protocols; Elastic File Share) 
Target Audiences 
e Business units 
e Students and professors 
e Application teams 
Use Cases 
e An application that requires storage for its database 
e Business units that need a shared file system for document storage 


e Organization requiring secure storage to backup and archive data for 
compliance 


Crimson Cloud: End-State Staff 


Crimson Cloud Develops and supports establishment of the 
laaS e Crimson Cloud Crimson Cloud service; provides Tier 3 support 
Product Owner for the cloud 
DevOps Engineer 
Crimson Cloud Bavelonare Expertise, best practices, and standards for 
PaaS P using PaaS will exist within the Cloud Center of 
: BA/QA Excellence, but there will be no centralized 
support team 
Crimson Cloud e Cloud Storage Senior Supports multiple storage options; works closely 
STaaS Engineer with DR product owner to ensure that storage 
e Cloud Storage and data are well designed 


Engineer 


Cloud Center of Excellence 


The Cloud Center of 
Excellence will 
provide best 
practices, standards, 
and technical 
oversight for all 
Harvard cloud 
services. This team of 
cloud architects and 
subject matter experts 
will provide the 
University with 
leadership, support, 
guidance, and training. 


Crimson Cloud 


Infrastructure as a Service Storage as a Service 


Provide oversight for all Cloud 
Services; includes laaS, 
PaaS, and SaaS models 


Educate and 
Provide Reference 
Cloud Center of Excellence Architecture for 
Secure Cloud 
Hosting 


=p 


Core Responsibilities Cloud CoE Team 


age eg Best Practices and Standards Technical & Cloud Architects 
to define ; Architecture Design Patterns Cloud SME 
governance for DevOps Training and Support Infrastructure SMEs 


sa and Vendor Analysis DevOps Engineers 


Compliant 
Cloud Services 


Communication and Outreach 
Knowledge Repository 


Provide Guidance and Support 
for Cloud Operations 


Cloud Operational Services 
Operations & Disaster Recovery 
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Cloud Services: End-State Staff 


Cloud Center of 
Excellence 


Cloud Architecture 
Manager 

Cloud Architect 
Infrastructure SME 
— part of Enterprise 
Architecture 


Experienced cloud architects in Harvard’s cloud 
deployments. Infrastructure SMEs are the best 
technical resources from 60 Oxford who want to 
remain focused on infrastructure (Windows, Linux, 
Storage/DB). Provides Tier 3 support for the 
cloud. 


IT Provider Services 


Network & Security Billing & Account Management 


Network & Security 
e Provides the foundational services required for any HUIT cloud 
application, as well as the core network and infrastructure security 
required for any application hosted in an external cloud 


Billing & Account Management 
e Integrates a partner organization into the consolidated billing process 
and high-level account management to cloud services provider; service 
also includes integration with HarvardKey identity and access 


management 


IT Provider Services: Network & Security 


This service (required for all cloud-hosted production applications) 
provides foundational services needed for any HUIT cloud app, including 
the core network required for any application hosted in an external cloud. 


Features 
e Fully-managed network connection to external cloud provider 
e High-speed, redundant network connectivity (Direct Connect for AWS) 
e Enables connectivity to defined cloud and/or on-premise resources 
e Routing and DNS support for cloud network integration 


Target Audiences 
e AlllaaS and PaaS offerings must use this service 
e Project teams with experience implementing and managing cloud 
infrastructure 


Use Case 
e Partner organization hosting applications fully supported by project team, 
but needing reliable connectivity and the ability to access internal Harvard 
resources (such as an internal database) 


IT Provider Services: Billing & Account Management 


This service integrates a partner organization into the consolidated billing 
process and high-level account management to cloud services provider. 


Features 


e Consolidated billing to produce greater savings and volume discounts 
e Governance over root-level cloud provider access 

e Integration with HarvardKey identity services for all accounts 

e Guidance on the mandatory tagging of resources to simplify 


Target Audience 
e All partners using Crimson Cloud services 


Use Case 
e Partner organization using either laaS or PaaS offerings 


IT Provider Services: End-State Staff 


Network and e No impact Supports DirectConnect and entire security 
Security architecture framework for AWS; builds upon 
current tools 


Cloud Operations Services 


Cloud Operations Services 


Cloud Operations Disaster Recovery 


Cloud Operations 


e A Suite of next-generation operational services and tools that provide 
application teams with the ability to manage, optimize, and assess 
production application workloads; will also extend and enhance on- 
prem capabilities to cloud resources 


Disaster Recovery 


e A Suite of service offerings to offer an appropriate level of DR for all 
applications that require it; primary offerings will be on-premise-to- 
cloud and cloud-to-cloud DR 


Cloud Operations Services 


Services and tools that enable app teams to manage, optimize, and 
assess production application workloads. Cloud Operations will extend 
and enhance on-premise capabilities to cloud resources. 


Features 
e Monitoring 
e Auditing 
e Logging 
e Scheduling 
Target Audiences 
e Application teams with compliance requirements 
e Production applications with high service levels and on-call requirements 
e App teams continuously optimizing for cost and performance 
Use Cases 
e An application team needing a means to identify the root cause of a 
service outage 


e Anapplication team that wants to spin up a new environment without 
creating downtime 


Cloud Operations Services 


Monitoring 

Advanced cloud monitoring will expand the current server monitoring service 
to cloud resources, with application performance and diagnostics information 
available as an optional service. 


Logging 

Current logging service will evolve and become a standard component for all 
cloud systems; tiered services will provide customers with options to meet 
their application or compliance requirements. 


Auditing 

Cloud auditing will become standard on all cloud systems, and will monitor 
cloud resources for a wide range of security-related configuration items, 
including security groups, S3 policies, and IAM users. 


Scheduling 

There is currently no known job schedule outside of Maestro that meets 
requirements; in the future, as new technology for the cloud is developed, we 
anticipate the service will use a specific cloud job scheduler. 


Disaster Recovery 


Design and support services required to recover applications or services 
to cloud resources. 


Features: 
e Cloud-to-Cloud: Applications already deployed in the cloud can recover 
within the same cloud provider or across different cloud providers 
e On-Premise-to-Cloud: Applications remaining on-premise will use 
CloudEndure to ensure application redundancy 


Target Audiences 
e Mission-critical applications 
e Enterprise applications 


Service Tiers 


*AWS-specific 


On-Prem-to-Cloud Not Available 


Backup & 
Restore 


Cloud-to-Cloud Multi-AZ* Cold Storage” Hot-Hot-Hot* 


Disaster Recovery 


SII 


Data Uses CloudEndure to build standby — Disaster at <1 hr 
Protection application for reconstitution at Harvard data = 
remote AWS data center center 
Multi-AZ Design services to use good cloud Individual data $ ~0 ~0 
Cloud design principles for horizontal center or AZ 
Architecture scaling and data protection using failure/ 
multi-AZ architecture degradation 
Cold Copy all data and configuration to Extended regional $$ <thr <8hr 
Storage secondary region for cold storage failure (staff 
(code+config+data) limited) 


Full environment build required and 
data can also be sent to secondary 


Hot-Hot-Hot Multi-region active production Regional failure or $$$$ <15 <30min 
running at full scale degradation min 


Cloud Operations Services: End-State Staff 


Disaster Recovery e DR Product Owner Manages DR communication, evaluates 
ongoing solutions, works with Crimson Cloud 
and storage product owners to define 
implementation approaches, manages testing 
across HUIT service teams 


Cloud Operations e Cloud Operations Manages, tracks, and reports on monitoring, 
Director auditing, and logging; partners with ITSM 
e Cloud Operations team on incidents to ensure that event 
Tools Developer reporting is appropriate; partners with 
e Cloud Operations application teams to meet application needs 


Optimization Engineer 


Application Teams: Overview 


This section provides an overview of the service impact associated with 
application teams or ITSM, including the impact of both the embedded 
DevOps engineer and the newly introduced services as part of the cloud. 

The examples that follow relate to HUIT’s ATS program, but in all cases the 
models work across application teams. 


Critical concepts for application teams include: 


Embedded DevOps 
Release management 
Developer toolset 
Application team structure 


ITSM process impact 


Cloud & DevOps Impact on Application Teams 


e Embedded DevOps engineers managed and cost-allocated to 
application teams 


e Common cloud services and multi-tenancy established 


e Scaled Agile Framework practices incorporated into the establishment of 
infrastructure 


e Embedded architects and the establishment of standard 
infrastructure patterns 


e Defined integration approaches for application connectivity within the cloud 
e QA automation and deployment integration 

e Automated, continuous deployment and changes to release management 
e Defined and published operational support models 


Sample Application Team: ATS 


Enterprise President & 
Applications FAS Provost Aurora 


ATS Project Services 
¢ Release Managers 


Communities of Practice: 
¢ Project Managers 

* CRMs 

* Product Owners 

¢ Business Analysts 

¢ Scrum Masters 


ATS Technical Operations 
¢ Data Architects 
¢ Software Architects 
Communities of Practice: 
¢ Software Developers 
* QA Engineers 
¢ Application Support 


HUIT Arch & Engineering 


* Embedded DevOps 
¢ Enterprise Architects 


Communities of Practice are accountable for best practices, with a full-time role and input in team rating 
28 


ATS: Release Management 
Release management evolves to provide an expansion of services to 


orchestrate automated release engineering processes. The auditable 
release engineering processes replace manual release validation with 


automated deployment packages. 


Automated Code Promotion 


Fork 


Release 
Management Dev Test QA Prod 


Release Management Processes 


ATS: Developer and Service Management Toolset Service 


This service provides a set of shared development tools teams across 
the University can use, including a set of common functions all groups 
use as part of the application development process. 


Features 
e Atlassian suite (JIRA, Confluence, Fisheye, Crucible, HipChat) 
Enterprise GitHub 
Data masking tool 
Automation tools (Puppet, Ansible) 
ServiceNow 
Target Audiences 
e Development teams 


Use Cases 
e A development team wishing to create a de-identified copy of 
production data to securely use in lower environments 
Teams that want to create a wiki for project collaboration 
A developer who wants to create a script to automate a repetitive 
manual process 


Application Teams: End-State Staff 


ATS Embedded 
DevOps 


Embedded DevOps 
for Other App Teams 


ATS Release 
Management 


ATS Developer and 
Service Management 
Toolset 


DevOps Engineer 


DevOps Engineer 


Release Engineer 


Toolset Lead 
Tools Engineer 


Teams remain as defined earlier, with the 
addition of embedded DevOps personnel 


Teams remain as defined earlier, with the 
addition of embedded DevOps personnel 


Provides broad release engineering support 
across ATS, and is integrated into the 
release management process; trains other 
release managers in release engineering 


Provide HUIT-wide development and 
service support tools, as well as tool 
patches and upgrades and operational 
support; assesses organizational fit and 
works with Cloud CoE for selection 


ITSM: Cloud Impact 


ITIL Lifecycle Cloud Impact 


Service Service portfolio Assess different cloud providers and their services. 
Strategy 
Service Service catalog, Introduce new services identified; establishing new 
Design service level features with these services will occur at fairly rapid pace. 
Service Change management Breaks the centralized control, but not the centralized 
Transition repository of information; autoscaling “just happens.” 
Service Event management, Increase pace of activity and response; reduce focus on 
Operation incident management, determination of problem and increase rapid creation for 
problem management, response. Automate process for event and request 
service requests response from ServiceNow to incite cloud change. 
Service Service reporting and Improve service metering capabilities. 


Improvement — quality 


End-State Tools (1/2) 


Tools are based on the current rapidly-evolving marketplace, and are 
therefore expected to be subject to change. 


DR CloudEndure DR purposes only 
Account Mgt Security Monkey AWS account management 
ae Cloud Allocation 360 Dynamic cost/billing analysis for both on-premise 
Billing 
and cloud 
Developer Github (Enterprise) Advanced version control system 
Toolset 
Developer VictorOps Automated incident paging 
Toolset 
Splunk Centralized and consolidated monitoring 


Logging 


End-State Tools (2/2) 


Monitoring 


Scheduling 


Release 
Management 


New Relic 


Zenoss Service 
Dynamics, Nagios 
Xl 


Nagios 


IBM Tivoli Workload 
Scheduler (Maestro) 


Bamboo (Atlassian) 


App monitoring, including dashboard and transaction 
introspection, automated alerting, and trend analysis 


Event management console 


Detailed system monitoring and alerting 


Automated script scheduling and batch processing 


Automated release management 
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Cloud & DevOps ‘Plan for the Plan’ 
service Transition Strategy 


Agenda 


e Purpose and Intended Outcomes 

e Goals for Cloud Service Transition 

e Cloud Change Principles 

e ATS Services: Current and Future State 

e ATS Transition 

e Infrastructure Services Current and Future State 
e Infrastructure Transition: Staff 


e Infrastructure Transition: Project Development 


Purpose and Intended Outcomes 


Purpose 


To outline principles and goals for cloud service transition, examine ATS 
as a case study, and discuss staff and process transitions for 
infrastructure at Harvard. 


Intended Outcomes 
e Understanding of our goals for cloud service transition 
e Knowledge of transition as instantiated within ATS 
e Awareness of infrastructure transition goals, processes, and options 


Goals for Cloud Service Transition 


Organizational Goals 
e Reduce inefficiencies through consolidating redundant roles 
e Create and implement a common Agile framework across organizations 
e Break down silos by developing common HUIT-wide standards and tools 


Infrastructure Goals 
e Decentralize operational support and create T-shaped professionals by 
implementing embedded architects and DevOps engineers 


e Develop cloud service offerings and transition from a traditional data 
center to the cloud 


Cloud Change Principles 


e Use ATS as lead adopters for all application teams 


e Define and communicate an embedded DevOps vision and its benefits for 
the team 


e Perform changes in small increments 


e Identify owners of organizational change within ATS to drive progress and 
remove obstacles 

e Define owners of process changes within the Cloud & DevOps 
program team 

e Provide training and actively communicate with ATS team members 
throughout the change process 


e Develop a regular, recurring reporting process to capture progress and 
identify course corrections 


ATS Services: Current and Future State 


The future state of ATS relies upon the addition of embedded DevOps 
resources, as well as close collaboration with data management 
integration services and the Cloud Center of Excellence. 


Current Services Future Services 


FAS & College Systems 
+ Embedded DevOps 


CA Systems 
+ Embedded DevOps 


OPP Systems 
+ Embedded DevOps 


Developer & Service 
Management Toolset 


ATS Transition: Defining an Approach 


Broad process changes are required, but they cannot be implemented 


all at once across the organization. 


Category Definition 


Development Consolidate and optimize 
Process ATS development, deploy, 
and support processes 


Embedded Introduce new roles and 
DevOps evolve support structures 
as a result 


Expected Lead Adopter 
Outcomes 


Create the ability to 
share resources across 
ATS 


Create the ability for FAS and College 
service teams to be Systems 
independent service 

teams 


The Development Process: Today vs. Tomorrow 


As a result of this change process, code development will also evolve. 


Today 


| : Harvard 
cy Ticket Data 
Deployment Queue ) Center 
| | Requests ~ 


Developer 
Staff 


Tomorrow 


Automated Code Promotion 


Code 
| | ————S > Repository —_——P Dev Test QA Prod 
Developer 


Release Management Processes 


ATS Transition: Development Process 


Our goal is to develop consistent project management, Agile, QA, and 
release management processes across HUIT. 


Assess and Plan Train Implement 


Work across teams to assess 
adoption viability and needs 


Train staff and prepare for 
implementing changes 


Identify upcoming releases to 
implement changes 


Develop strategy and 
implementation plan for 
rollout of changes 


Evolve 


Build and implement 
automated QA functionality 


Build and implement release 
engineering functionality 


Build and implement auditing 


— SP ——? Implement changes while 


working through Agile 
delivery process 


Mature 


Continue to guide new Agile 
teams and expand Agile skills 
Incorporate program 
increment planning 


ATS Transition: Embedded DevOps 


Introduce embedded DevOps and Jetstream teams, accompanied by 
operational support changes. 


Plan Align Staff Train 
Define vision and expected Align Jetstream staff to the Joint completion of Cloud & 
outcomes FAS and College System DevOps training program 


Define development strategy ——— Portfolio ——? ATS team trains Jetstream 

for Jetstream creation Create formal relationship on application development 

Develop training plan with the ATS team and the methodology/architectures 
Jetstream team for FAS/College Systems 


Implement Evolve 
Migrate 210 FAS servers using lift- Enhance monitoring and notification 
and-shift approach for migrated applications 


Transition Jetstream team into ——p_ Validate that training curriculum is 
embedded DevOps for FAS appropriate 


Transition operational support for Find opportunities to refine process 
any migrated system Replatform applications 


Infrastructure Services: Impact to the Service Catalog 
Infrastructure services will evolve, remain the same, or be deprecated. 


Current Services Future Services 


Crimson Cloud 


—_————__________—__—_» 
Storage as a Service 
Data Protection 
Snapshots, Self-Healing 
Networking 


+ Increased Extranet Connectivity 


Crimson Cloud 
laaS + PaaS 


Cloud Center of Excellence 


Infrastructure Transition: Defining an Approach 


Infrastructure as an organization does not exist in the future state; the 
overarching goal is to build the skills and supporting services to create 
the capability to enable DevOps. 


Staff Transition sysadmin staff Build expertise and Jetstream team, Cloud 
into new roles needed for capability to support & DevOps program 
the cloud cloud services staff 

Project Complete associated Completed Infrastructure teams 

Development projects with Infrastructure — infrastructure services (varies) 
to enable future cloud that utilize the cloud 


services 


Infrastructure Transition: Staff 


Create a clearly defined plan for transitioning staff 
e Incorporate Jetstream transition to app teams as migrations take place 
e Include backfill of cloud program staff as they migrate into different roles 
e Identify a DR product owner 


In support of migrations, the Cloud & DevOps program will drive transition 
of Infrastructure staff to enable cloud skills 


e Lift and shift 
o. Skills to transfer existing applications to cloud environments 
o Application awareness 
o Replatforming within application teams 

e Replatforming 
o Skills to assess existing application designs 
o Skills to refactor applications to conform to cloud standards 
o Skills to QA revised applications 


Infrastructure Transition: Staff 


Training paths for Jetstream and program teams have established 
progressions and criteria for end-state roles. However, team members 
can also work towards meeting criteria to move into other roles. 


Cloud Program Sample Criteria Role 
Default 


eae Devel t | Cri Cloud | 
evelopmen rimson Clou 
(Replatform and err al | | 
Crimson Cloud) Cloud Storage StaaS 
Cloud 
Atlassian ATS Toolset 
Infrastructure 
Staff: Monitoring Config Ops Tools 
SOC 
Usage Analytics Ops Optimization 
Cloud design CoE Architect 
Infrastructure CoE SME 


Cloud Jetstream Beenie 


Teams Default 
(Embedded ——————® Cloud config DevOps Engineer | 
cloud DevOps 
administration) Engineer Specific skills Independent roles 


associated with role (Release Engineer, DR) 


Infrastructure Transition: Cloud Jetstream Teams 


Create small teams of cross-functional infrastructure teams to embed 
with application teams to support the portfolio in the cloud. 


Staff Assessment Implement Train 

Review staff database for Create Jetstream team for Train Jetstream teams on 
staff to move into next Cloud HUIT organizations assigned applications 
Jetstream — > Distribute application/server —— Train Jetstream teams on 
Assess operations impact portfolio to align with AWS fundamentals 
and transition activity as Jetstream teams 
needed 

Execute Embed Support 
Validate discovery material Embed staff into application Work with app teams, Cloud 
with Jetstream teams teams CoE on replatform plan 
Build skills through hands-on Provide DevOps engineering Provide ongoing training and 
active migration of mentorship through the cloud mentorship through cloud 


applications program team program team 


Infrastructure Transition: Cloud Program Team 


Migrate staff into a central cloud migration team to replatform 
applications and learn development skills. 


Staff Assessment Implement Train 

Review staff database for Identify app migrations and Train staff on AWS 
staff to move into the cloud mentor for staff fundamentals 
program team —— Integrate into cloud program 9 = Train staff on Agile 
and transition activity as fundamentals 
needed 

Execute Transition Support 
Build skills through hands-on Transition staff into Crimson Evolve training based on 
migration and replatforming Cloud role or appropriate role market changes 
Build advanced skills through Develop product support Establish new cloud services 
the establishment of new skills associated with 


applications and tools establishing a service 


Infrastructure Transition: Crimson Cloud 


Crimson Cloud will be structured as a Pe snp ts Anign Archies A tp: ny Prod 


cl andine a ravops 


product development and support team: tt me 


e Strong product owner to define service 
strategy and manage development iat Rinne 
e Development and DevOps engineers to ihe ae = = 
implement self-service capabilities 
e Close work with Cloud CoE to ensure Soot, rac aese nance ea oeaas oe 


mie dreetiaeertir “ree ore nai restate sing fo iden sing Tesources: 


services are aligned to best practices 
and standards 


steps 4: Train <r cae on faaS and aye 
Desaipsan Dyess 3S service affanng 


Why is this important? 6 ge ag a a 
Crimson Cloud will own all aspects of the 


cloud hosting service and be responsible for PaaS Step 5: Transfer the vendor jaaS Step &: Transition Crimson 
delivery of cloud services. Scr tecreremgeeenoe Se 


jonraation tal wail te supporting the crewscin 


Crimson Cloud Working Group 

Regular sessions with user community will 
gather feedback on features, functionality, 
issues, and concerns. 


Infrastructure Transition: Project Development 


The projects below require coordination between the cloud and 


infrastructure teams. 


Scheduler 
Service 


Data Protection 


Storage 
Strategy 


Billing 


Cloud Network 
Firewall 


GoldenGate 


Network 
Simplification 


Evaluate Maestro for cloud viability and migrate or 
replace with cloud alternative 


Evaluate risk and approach for providing ongoing 
protection of Harvard data in the cloud 


Integrate storage strategy and cloud StaaS 
definition into coherent project implementation 


Assess long-term viability of Slash system and 
either migrate or replace with alternative 


Establish centralized firewall approach to protect 
cloud assets 


Establish a database integration service 


Migrate the existing complex ACL and network 
layered approach into a simplified approach that 
takes advantage of cloud technologies 


Scheduler service that allows direct 
application team scheduling updates 


Document outlining protection methods for 
Harvard data; establish associated services 


Storage strategy document and associated 
implementation plan 


Billing solution supported by multiple staff 


Cloud network firewall that allows Level 4 
data to be hosted in the cloud 


Direct database replication service 


Simplified network structure and associated 
service updates 


Operations 


Data Protection 


Storage and 
Archive 


Cloud and DevOps 


Extranet 
Connectivity 


Database 


Data Center 
Networking 


Infrastructure Transition: Project Development 


All projects should follow standard project methodology. Below are 
extra steps to coordinate projects with the cloud team and architecture. 


Create Team 


Identify project managers for 
each project (scrum masters) 


Identify team members from 
Infrastructure and cloud 
program to work with scrum 
master 


Gather Requirements 
Identify any cloud 
dependencies and scheduling 

bs needs for the project 


Identify external consulting 
needs where resource 
contention exists 


—» 


Plan and Design 


Define vision and approach for 
future state of technology 


Review design with cloud 
leadership 


Validate impact to funding 
model 


Implement 


Execute and validate project 
against requirements 


weet HARVARD UNIVERSITY 


W4S|% Information Technology 


Cloud & DevOps ‘Plan for the Plan’ 
Cloud Migration Approach 


Agenda 


e Purpose and Intended Outcomes 
e Current Challenges 
e Migration Approach Options 
o Replatforming 
o “Lift and Shift” and Jetstream 
o Cloud Migration Consultants 
o Organic Transition 


e Recommendations 


Purpose and Intended Outcomes 


Purpose 


To highlight observations from HUIT’s current migration approach, 
describe alternative approach options that were considered, and provide a 
recommendation for a future migration approach. 


Intended Outcomes 
e Understand the current migration constraints 
e Describe alternative migration approach options 
e Review and compare migration approaches 
e Outline recommendations 


Current Challenges 


Replatforming has encountered multiple constraints, impacting our 
ability to meet program goals. 


Velocity of migrations not sufficient to meet the program goals 
e Current process migrates approximately 2-3 apps per two-week sprint 


e Speed of migrations has increased as team gains more experience, but 
still not enough to meet program goals 


Identifying applications to migrate is challenging 
e Developing a backlog of applications has been difficult 


Training DevOps engineers takes substantial time 


e Aconsiderable amount of time will be required to train the number of staff 
necessary to support the end state 


Currently unable to migrate applications with Level 4 data 


e Infrastructure changes required to support Level 4 data have not yet 
been implemented 


Migration Approach Options 


There are four identified approaches for migrating code into the cloud 
environment. Their commonalities and differences are defined below. 


Replatform Optimized for Cloud program Evolving Crimson Cloud toolset, 
cloud team Crimson Cloud migrated applications 
toolset 
Lift and Shift Unchanged Cloud program CloudEndure Codebase in cloud, 
team ready for independent 


replatforming efforts 


Migration Optimized for Consultants Proprietary Codebase in cloud, but 

Consultants cloud ongoing support is 
dependent upon 
consulting 

Organic Varied Application team Varied “Have and have not” 


silos, distributed 
approaches 


Migration Approach: Replatforming 


The Process 

Applications are updated to maximize the utility of cloud features, 
and are then deployed to the cloud by this team. All new 
applications follow this process. 

Staffing 

Cloud & DevOps program team platforms all applications 


Benefits 
e Applications replatformed for the cloud see improvements in 
quality, improved alerting, and increased reliability 
e Crimson Cloud tools — A library of reusable toolsets to 
maximize automation abilities of all application teams 
Challenges 
e Limited number of DevOps engineers available to support 
replatforming; currently 8 engineers included in Wave 1 
e Changes to the application infrastructure, requires time and 
work from application and DevOps teams 
e Ongoing operations support model is not sustainable 


Engagement 


Replatform 
Migration 


Testing/ 
Validation 


Migration Approach: Lift and Shift (1/2) 


The Process 

The application, infrastructure, and cloud program teams jointly identify 
dependencies, plan, and migrate apps. Application teams validate the 
migration was successful; embedded team members provide ongoing support. 


Staffing 


Infrastructure team staff will be assessed and split into support teams to drive 
migrations using CloudEndure. 


Jetstream Application Teams 


~ Testing/ 


Engagement -® Discovery -® Migration Validation 


I 

| 
! | 
| 
Re-platform | 
| (as required) 
| 
! | 
| 
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Migration Approach: Lift and Shift (2/2) 


Benefits 
e This approach reduces migration time, allowing us to decommission on- 
premise servers more quickly and reduce hosting costs 
e Allows existing staff to manage migrated servers with limited knowledge of 
cloud technologies — this accelerates migration but extends timeframe to 
both train and replatform application 


Challenges 
e The discovery process to determine application dependencies is intensive 
and potentially time-consuming 
e Replatforming is dependent upon the application team prioritization and 
support for embedded infrastructure teams 


Jetstream Application Teams 


~ Testing/ 


Engagement - Discovery -® Migration Validation 


| 
! | 
| 
Re-platform | 
| (as required) 
| | 
| | 
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Migration Approach: Jetstream 


“Jetstream” teams are small, cross-functional groups made up of 
sysadmins and DBAs from the SOC. These teams will have all of the skills 
necessary to support applications while also migrating them to the cloud 
using “lift and shift” methods. Benefits of Jetstream teams include ... 


Provide app support and migrations 
e Performed in parallel 
e Part of the application team 
e Build application culture first 


Linux SA 


Increase migration velocity 
e “Like for like” migrations 


Cloud exposure to more staff, faster 
e Effectively extends our training time 


Gradual expansion of skills 
e Teams develop replatforming skills 


Migration Approach: Cloud Migration Consultants 


The Process 
Hire a third-party vendor to migrate applications to the cloud. The vendor will 
define and manage the process based on their best practices and methods. 


Staffing 

Migration will be staffed by vendor, but will still require significant help from 
application teams, network, and security to reconcile dependencies and 
integration into Harvard's network. 


Benefits 
Delivery-focused vendor will accelerate the timeframe. 


Challenges 


e HUIT loses in-house skills to support migrated applications, 
requiring us to rely on the vendor for support until HUIT is able to hire in 
necessary skillsets 


e Overall increased cost for migrating applications 


Warning: Vendor-Dependent Process! 


Migration Approach: Organic Transition (“Do Nothing”) 


The Process 

Partner organizations are responsible for migrating on-premise apps and 
resources, with schedule controlled by partner organizations. 

Staffing 

Partner organizations are responsible for staffing. 


Benefits 
e No need for a dedicated Cloud & DevOps program 


Challenges 
e Inconsistency in app migrations lead to information silos 
e Increase in security risk due to inconsistent cloud deployments 


App Team A App Team B App Team C App Team D 


On-Prem A B Cc D 


| ! | 


Recommendation: Hybrid Approach 


After analyzing these migration approaches’ strengths and weaknesses, 
we have decided to adopt a consolidated approach that offers the increased 
velocity and collaboration of lift and shift with the flexibility and cloud 
optimization of replatforming. The default approach is lift and shift, but there are 
instances where replatforming is appropriate. 


(nn Lift and Shift 
NO 


Team Constraints 
Technical Constraints 
Engagement —> Criteria Vendor Constraints 
Data Constraints 
Licensing Restrictions 


Replatform 


Benefits of a Hybrid Approach 


Expands the number of staff that participate in the cloud program 
¢ Lift and shift enables the SOC and application teams to learn and apply 
basic cloud and AWS technologies 
¢ Program team can focus on reusable artifacts and future-state services 


Initial lift-and-shift phase provides agility and flexibility to app teams 

¢ Offers a streamlined approach to provision resources faster in order to 
address technical debt (e.g. outdated operating systems) 

¢ Provides basic assurance that an application can run “as is” in the cloud; 
eliminates “too many changes at once” 

¢ Reduces network complexity 

¢ Enables teams to plan for replatforming according to partner timelines and 
with the support of an embedded resource 


HUIT can save costs and meet program goals with current support levels 
¢ Allows for increased pace of decommissioning on-premise infrastructure 


¢ Application support model remains “as is” until Support resources are 
embedded into the team 


Staff Transitions: Lift and Shift/Replatforming 


Please see the handouts for details of the staff transition approach. 


Migrate First (“Lift & Shift”) Staff Transition Approach 


ee Perform Staff Assessment 


Assess sls. cometences and cultura tof Inastuctre staff 
‘One by "hes fi for Cloud & DevOps program cansiacy oF DevOps engineering 
Greate raoontry of rare sao erage fue hart and Raabe resoure slannng 


2a Create "MiniSOC” Teams for HUIT Organizations! o 
Programs 


+ Base on sal ascesement, ate eral soss incon 
infrastructure ame 


‘sign oan oronization (Le ATS) orsubset ofan osanzaton— 
(ce ATSIFAS) 
+ Move HR reporting for hese taf trom nastctire to 
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‘Train MiniSOC Teams on AWS Fundamentals 


+ Provide MiiSO feome with undamentat loud traning vis 
‘Sneeom and nb exreane 

+ Propae ams to navigate the AWS console and provide proper 
‘permona! suppor fr R-and-sn” migrmtions 


Distribute Application/Server Portfolio to Align with 
MiniSOC Teams 
~ Review CMDESlash systems to create comprehensive 
‘nplcaleniserver ventory 
‘Segment inventory to align wth new team stuctures 
+ Assign appsservers o teams 


‘Train MiniSOC Teams on Assigned Applications 
= Aptction tas werk wth sig MiySOC m oreew 
‘application portolo and associated technical architectures 
Aplications rin MISC teams on appieation Lp! 
“aeeeopmen Mecycle processes and ncide them n nonin 
project work to further Bar understaning 


‘MiniSOC Teams Migrate Assigned Applications ‘Ongoing Operational Support for Assigned Apps 
| Perform dmcovery ane! migrate "andar nape Update Supoort Services team to help support processes to 
= Werk wth apsitian team for validation ad mesmmieioning licaun for new operational supper suture 
ol on-premise inasucture + Me/SOC teams provi Ter 2 and 3 operational suppor tor 
= Document design ana update service now PR setires appears my arden wih migrabon responses 


‘Train MiniSOC & Application Teams for Replatforming and DevOps 
+ Bath appicabon and Min teams compte Yainng to prepare them fr rglatonmang appicabons.n her pote and perermeg as il tac 
raineers 
= Re-assnes name! ality t perform ongoing work as DevOps engineers 


Distribute MiniSOC Staff Among Application Teams 
+ Assign new Devos engineers to teams 


Change HR reporting relations rom Architecture & Engineering to the team 
© Covoeate stat 


Application Teams Manage Replatforming as Needed 


Aoicaton teams wor wth architects nthe Cloud Centro! Evenlence la design replationnrg fo 
/Apicalion teams wort to schedule, manage. and implement relaforing,leverasing embedded DevCps sat 


‘Application Teams Provide Operational Support Post-Replatforming, 
-Appicaton tnams and embeded Devs staf perform onging operational spar ane mainnn clne-teptoyed applications 
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PHASE A: PLANNING 


‘Wave 0: Seed Statt 
Enrty sareary 2005 (comroete) 


Neare roc. 10 eC mNSRTTERES epee O Core Taare” DAaTT'g IF ANT Comet are reponse oe 
Oey or wecing fourcetcre proce scteties ani tactrcsoges remit lo mgrate aoe 


+ Warten tern reefer eat 
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PHASE B: DESIGN & MIGRATION 


Choxad Operations Tea Liat 
DevOns Putiorm Team Engreer 


Wave 1: Pilot Tools, Establish Processes, and Migrate Applications 
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are atte Wave Lapecat ategremert. These sat? 
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CLOUD & DEVOPS Staff Transition Workflow 


This guide to the Cioud & DevOps team launch process provides an overview of the full timeline and guidelines for successful staff transitions. 
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Repeatable Process (3-4 Weeks) Assigning Statt to Waves 
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PHASE C: TRANSITION 
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Ongoing: Transition Prior Waves to Core and/or Service Teams 
43 each row mre bepra wale recy bon ser awe wb Partin bem SO hur St to core and/or service tearra Te ermure wrect! baraiicn Dewees eaves tere ol be am overiap percd uring ahech otha stat wi prove mentorhe te resreng reas 


Other hey acteties occuring ©) oatrecton we) mare MCLE 


+ AR essen ef salary are erate fot equity 


Space glancing actwies Yo uccomrredate Warsiers 


Additional Effort: Staff Communication and Outreach 
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Migration Workflow: Lift and Shift 


Amazon Data Centers 
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